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Education Consolidation and Improvement Act - Chapter 1 

REPORT OF THE CHAPTER i 
SUSTAINED EFFECTS STUDY 
1986-87 

ABSTRACT 



Reason for the Study: According to ECIA Chapter 1 guidelines school districts 
receiving Chapter 1 funds are to conu 'c. a study of sustained effects once 
every three years and use the results in the planning of Chapter 1 programs. 
Consequently the study addresses three questions. First, what effect did the 
1985-86 school year have on the achievement levels of pupils in the CLEAR and 
Reading Recovery Programs? Second, what was the effect of the time period 
between April, 1986 and October, 1986 on the achievement levels of pupils in 
the programs? Third, what effect did the 1986-87 school year and the 1986-87 
CLEAR program involvement have on achievement levels of the 1985-86 
compensatory program pupils? 

Method; The study focused on the extent to which reading achievement effects 
of Chapter 1 programs were sustained over a period of time as measured by the 
Comprehensive Tests of Basic Skills (CTB/McGraw-Hill, 1981). The study 
employed a f ali-spring-f all testing design which began with the 1985-86 school 
year. A subsample of pupils were studied using a fall-spring-fall-spring test 
design, which provided a longer look at sustained effects. Pupils included in 
the study met the following criteria. First, the pupils participated in the 
CLEAR or Reading Recovery program in a district school during the 1985-86 
school year. Second, the pupils were in grades one through seven during the 
1985-86 school year. Third, each pupil attended at least 80% of the program 
days between pretest and posttest. Fourth, pupils were promoted to the next 
grade at the end of the 1985-86 school year. Fifth, complete pretest, 
posttest, and sustained effects test data were collected on the pupils. A 
second "sustained effects" test from the April, 1987 districtwide testing was 
obtained for a subsample of the pupils who met the above criterion. 

Results: Of the 4263 pupils eligible for the sustained effects study, 78.2% 
(3338) met the five criteria and were included in the study, and 70.3% (2989) 
had a fourth test score. The average pretest-posttest NCE gain for all pupils 
in the Chapter 1 CLEAR piogram was 6.5 points. When the sustained effects 
testing was done in September, 1986 the average NCE score dropped 5.3 points, 
for an average NCS gain of 1.2 from pretest to sustained effects test. Of the 
3338 pupils in the study, 36.2% (1134) maintained or exceeded their posttest 
NCE level on the September, 1986 sustained effects test. 

The average pretest-posttest NCE gain for pupils in the subsample who had 
an April, 1987 test scove was 6.7 points. When the sustained effects testing 
was done in April, 1987, the average NCE score dropped 0.6 points, for an 
average NCE gain of 6.1 from pretest to April, 1987 sustained effects test. Of 
the 2989 pupils in the subsample, ^7.9% (1432) maintained or exceeded their NCE 
posttest level on the April, 1987 sustained effects test. 
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Summary ; The sustained effects design included a fall-spring-fall testing 
schedule with an additional spring test for a subsample of pupils. In the 
i985-86 Chapter 1 CLEAR and Reading Recovery programs 3338 pupils were in the 
evaluation sustained effects group, 2989 in the subsample. Those pupils took 
each of the 3 or 4 scheduled tests, met the 80 percent attendance criterion of 
instructio.ial days, and were promoted to the next grade level. 

The results of the sustained effects study indicate some decline in all 
grade levels in reading during the summer months, with sixth grade having the 
smallest decline. The greatest net gains from Fall 1985 to Fall 1986 were 
noted for first grade Reading Recovery, and 2nd, 3rd, and 6th grade CLEAR. 
Pupils from these grade levels substantially improved their Fall 1986 
achievement levels by Spring 1987. 
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Education Consolidation and Improvement Act - Chapter 1 



REPORT OF THE CHAPTER 1 
SUSTAINED EFFECTS STUDY 

July 1987 



ECIA Chapter 1 evaluation guidelines require scho 1 districts receiving 
Chapter 1 funds to conduct a study of the sustained effects of Chapter 1 
programs. The study is to be conducted once every three yaars and the results 
are to be used in the planning of Chapter 1 programs. 

In order to examine the extent that the reading achievement effects of 
Chapter 1 programs were sustained over time, the Columbus Public Schools 
employed a f ali-spring-f all testing design. This design shows the degree to 
which the pupils who participated in the compensatory programs remained at 
least at the same level from spring of the treatment year until fall of the 
next school year. Additionally, test scores from the April 1987 districtwide 
testing were obtained allowing a fall-spring-fall-spring test design for a 
subsampie of the students. This provided a look at the reading achievement of 
students approximately one and one-half years after they were first served by 
Chapter 1 during the 1985-86 school year. 

Pupils from the Compensatory Language Experiences and Reading (CLEAR) 
program and the first grade Reading Recovery program were ini^luded in the 
study. The purpose of the CLEAR program is to provide assistance to 
underachieving pupils in grades one through eight so they may improve their 
language and reading skills. The purpose of the Reading Recovery program is to 
provide early intervention to underachieving first grade pupils who appear 
unlikely to learn to read successfully without intensive instruction. 

This report addresses three questions. First, what effect did the 1985-86 
school year have on the achievement levels of pupils in the CLEAR and Reading 
Recovery Programs? Second, what was the effect of the time period between 
April, 1986 and October, 1986 on the achievement levels of pupils in the 
programs? Third, what effect did the 1986-87 school year and 1986-87 CLEAR 
program involvement have on achievement levels of the 1985-86 compensatory 
program pupils? 

Method 

Pupils 

The pupils included in the sustained effects study met the following five 
criteria. First, the pupils participated in the CLEAR or Reading Recovery 
program in one of the Columbus Public Schools during the 1985-86 school year. 
Non-public program pupils were not included in the study because of the 
procedural difficulty of locating these pupils the following school year. 
Second, the pupils were in grades oiie through seven during the 1985-86 school 
year. Third, each pupil attended at least 80% of the program days between 
pretest and posttest. This meant that each pupil attended at least 108 of the 
135 program days. Fourth, pupils were promoted to the next grade at the end of 
the 1985-86 school year. The sustained effects test was given as part of the 
pretest procedures for the 1986-87 school year. Thus pupils were given the 
level of the test specified by the Chapter 1 evaluation design. Pupils who 
were retained took a lower level of the test than did promoted pupils. A study 
of the raw scores of retained pupils indicated that the test level was somewhat 
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low for these pupils and that a "ceiling effect" had occurred^ For this reason 
the scores of retained pupils were dropped from the study. Fifth, complete 
pretest, posttest, and suscained effects test data were collected on all of the 
pupils. In addition, a second "sustained effects" test was obtained from the 
April, 1987 districti^ide testing for a subsample of students in the study. 

Table 1 provides a description of the effect of applying these five 
criteria. The first set of figures shows the number of pupils at each grade 
level in the CLEAR and Reading Recovery programs who met the first three 
criteria for inclusion in the study. The second set of figures shows the 
number of these pupils having sustained effects scores. The third set shows 
the number who met all the criteria and were included in the study. The 
remaining figures show the subsample of study pupils having an April, 1987 
districtwide test who were served by CLEAR the 1986-87 school year and those 
who were not. An examination of the first two sets of figures reveals that 
sustained effects scores were collected on 83.7% of all program students 
eligible for inclusion in the study. A comparison of the first and third sets 
of figures shows that of all the 1986-87 program pupils eligible for the 
sustained effects study 78.22 met the five criteria and wer<^ included in the 
study. Furthermore 70.3% had a fourth test score, the April, 1987 dlstrictwide 
test. 

Test Instruments 

CLEAR and Reading Recovery pupils were administered the Comprehensive 
Tes ts of B asic Skill s (CTB/McGraw-Hill, 1981). This test series has empirical 
norms for fall and spring, established October 6-10, 1980, and April 27 to May 
1, 1981. The subtest, test level, and form used at each grade level are listed 
in Table 2. 

Procedur e 

In the Fall of 1986, the compensatory education Program Involvement File 
(PIF) for 1985-86 was used to identify pupils eligible for inclusion in the 
sustained effects study. (To be eligible the pupil had to meet the first three 
criteria given in the "Pupils" sectioa of this report.) A printout listing 
necessary pupil and testing i-^formation was produced for each school. 

A compensatory education teacher assigned to a school served as the 

sustained effects liaison person for the building. The printout and the 

necessary test materials and instructions were sent to the liaison person. The 

liaison person was responsible for collecting and reporting test data for 
pupils listed on the printout. 

The sustained effects test data reported by the the liaison persons were 
entered into the PIF system. The data were edited, and derived scores (grade 
equivalent, percentile, standard score, normal curve equivalent) were applied 
for the pretest, posttest, and sustained effects tests. The file was matched 
against the 1987 dlstrictwide test file in order to obtain all available test 
data and then matched to the 1986-87 PIF to determine pupils" subsequent year 
program involvement. This resulted in a data file for CLEAR and Reading 
Recovery pupils in the sustained effects study. From this file, normal curve 
equivalent (NCE) means, standard deviations, and frequency distributions were 
computed for the pretest, posttest, sustained effects, and dlstrictwide test. 
Distributions of percentile ranks were established. 

ERIC 6 

EVALSRVCS/P530/SUSRPT01 



Table 1 

Number of Pupils Meeting Various Criteria 

for Inclusion in the Sustained Effects 
Study Reported by Program and Grade Level 



1985-86 
Grade/Treatment 



Pupils With Pretest Scores, 
Posttest Scores, & Met 
Attendance Criteria 



Pupils With Sustained 
Effects Scores 



Total 



Pupils Who Were 
Promoted and Included 
in the Study 



1 


CLEAR 


103 


66 
103 






Recovery 


141 


55 
93 


2 


CLEAR 


859 


717 


630 


3 


CLEAR 


814 


698 


667 


4 


CLEAR 


828 


709 


688 


5 


CLEAR 


718 


609 


597 


6 


CLEAR 


614 


508 


460 


7 


CLEAR 


191 


161 


148 



Pupils in the Study with 
4-87 Districtwide Test Score 



4^68 



3571 



3338 



1986-87 
CLEAR 
Se rvic e 

22 
5 

193 
346 
363 
293 
77 
6 

1305 



No 86-87 

CLEAR 
Se rvic e 

20 
53 

385 

245 

279 

241 

337 

124 

1684 



Total 

42 
58 

578 

591 

642 

534 

414 

130 

2989 



EVALSRyCs/P530/SU^ABL01 

ERIC 



8 



Table 2 



Testing Schedule for the Reading 
Sustained Effects Study 



1985-86 
Treatment 
Grade 


1986 
Sustained Test 
Grade 


P T"0 r O C r 

(10-85) 
Test/Level/Form 


Pos t tes t 
(4-86) 
Test/Level/Form 


Sustained Effects 

Test (9-86) 
Test/Level/Form 


Districtwide Test 

(4-87) 
Test/Level /Form 


1 


2 


Total Reading/6/U 


Total Reading/C/U 


Comprehension/D/U 


Comprehenslon/D/V* 


2 


3 


Comprehension/D/U 


Comprehension/D/V* 


Comprehension/E/U 


Comprehension/E/V 


3 


4 


Comprehension/E/U 


Comprehension/E/V 


Comprehension/F/U 


Comprehension/F/V* 


4 


5 


Comprehension/ F/U 


Comprehension/F/V* 


Comprehension/G/U 


Comprehension/G/V 


5 


6 


Comprehension/G/U 


Comprehension/G/V 


Comprehension/G/U 


Comprehension/G/V 


6 


7 


Comprehension/G/U 


Compr>.ihension/G/V 


Comprehension/H/U 


Comprehension/H/V* 


7 


8 


Comprehension/H/U 


Comprehension/H/V* 


Comprehension/H/U 


Comprehension/H/V 



^Estimated by administration of abridged form V. 



10 



9 

ERJClSRVCS/P530/SUSTABL01 



5 



Results 

The following is a report by grade level of the reading scores of the 
1985-86 CLEAR and Reading Recovery pupils. Figures 1-9 and Tables 3-5 pertain 
to the 3 measurements ot the 3338 sustained etfects study pupils. Figure la - 
9a and Tables 3a - 5a pertain to the subsampJe of 2989 pupils who had 4 
measurements. 



n 
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AVERAGE 
r>CE 



52 
48 
44 
40 
33 
32 
28 
24 
20 
16 
12 
8 
4 
0 




1985-86 CLEAR {n=55) 



10-85 4-86 9-86 

TEST DATE 



•• 52 
" 48 
•• 44 
•• 40 
■■ 36 
■• 32 
■• 28 
•• 24 
•• 20 
•• IS 
■• 12 
8 

-• 0 



Figure I. A comparison ot the average pretest, posttesc, and sustained 
etfects NCE scores for first-graders in the 1985-86 Chapter 1 
CLEAR program, 

nEAR^^Uri^^^ pretest-postteot NCE gain for first-graders in the Chapter 1 
t.^Jlr^ ""^^ sustained effects testing was done in 

September, lySb the average NCE score dropped 4.1 points. The average NCE gain 
from pretest to sustained effects test was -2.2 points. of the 55 CLEAR 
first-graders in the study, 40.0/i {22 pupils) maintained or exceeded their NCE 
level on the sustained effects test. 

On the pretest 52,7% (29) of the 55 pupils in the Chapter 1 CLEAR program 
scored above the 36th percentile. On the posttest 58.2% (32) scored above the 
36th percentile and 49. U (27) scored above on the sustained effects test. 
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p. 



AVERAGE 

Nce 



52 
48 
44 
40 
36 
32 
28 
24 
20 
ie5 
12 
8 
4 
0 




1986-87 CLEAR (n»22) 
•a- NO 86-87 CLEAR (n»20) 
'■■ TOTAL (n-42) 



10-85 4-86 9-86 

TEST DATE 



4-87 



1 



52 
48 
44 
•■ 40 
•■ 36 
•• 32 
■• 28 
•• 24 
20 
■■ 16 
•■ 12 
•• 8 
■• 4 
■■ 0 



Figure la. A comparison of the avera^-.e pretest, posttest, 
sustained effects, and districtwl.'s NCE scores 
for tirst-graders in the 1985-86 Chapter 1 CLEAR 
program. 

For the subsample of first-graders in the 1985-86 Chapter 1 CLEAR program 
having a fourth (districtwide) test score, the average pretest-posttest NCE 
gain was 1.7 points. When the districtwide testing was done in April, 1987 -he 
average NCE score dropped 2.4 points (3.1 for pupils not receiving 1986-87 
CLEAR program service, 1.8 for those served). The average NCE gain from 
pretest to districtwide test was -0.7 (2.0 for pupils not receiving 1986-87 
CLEAR, -3.1 for those served). Of the 42 first-graders in the subsample, 40.5% 
(17) maintained or exceeded their posttest NCE level on the districtwide test. 

On the districtwide test 52.4% (22) of the 42 pupils in the subsample 
scored above the 36th percentile (70.0% [14j of the 20 pupils not receiving 
1986-87 CLEAR program service, 36.4% [8J of the 22 served). 
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1 ST GRADE READING RECOVERY 



AVERAGE 
NCE 



52 1 




48 ■ 




44 ■ 




40 ■ 




36 • 




32 ■ 




28 • 




24 ■ 




20 • 




16 • 




1 2 • 




8 • 




4 ' 


• 


0 • 






•♦■ 1985-86 READING RECOVERY 
(n>93) 



■4- 



10-85 4-86 

TEST DATE 



9-86 



52 
■■ 48 
■■ 44 
•• 40 
■• 36 
•• 32 
•■ 28 
•• 24 
•■ 20 
16 
12 
8 
4 
0 



Figure 2. A comparison of che average pretest, posttest, 
and sustained effects NCE scores for tirst-graders 
in the 1985-86 Chapter 1 Reading Recovery program. 

The average pretest-posttest NCE gain for first-graders in Chapter 1 
Reading Recovery was 10.0 points. When the sustained effects testing was done 
in September, 1986 the average NCE score dropped 4.7 points. The average NCE 
gain from pretest to sustained ef. :s test was 5.3 points. Of the 93 Reading 
Recovery first-graders in the study, 38.7% (36 pupils) maintained or exceeded 
their NCE level on the sustained effects test. 

On the pretest 24. 7X (23) of the 93 in Chapter 1 Reading Recovery scored 
above the 36th percentile. On the posttest 50.5% (47) scored above the 36th 
percentile and 24.7% (23) scored above on the sustained effects test. 
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AVERAGE 
NCE 




10-85 4-86 9-86 4-87 

TEST DATE 



figure 2a. A comparison of the average pretest, posttest, 
sustained effects, and districtwide NCE scores 
for tirst-graders in the 1985-86 Chapter 1 
Reading Recovery program. 

For the subsample of first-graders in 1985-86 Chapter 1 Reading Recovery 
having a fourth (districtwide) test score, the average pretest-posttest NCE 
gain was 12.7 points. When the districtwide testing was done in April, 1987 
the average NCE score dropped 4.6 points (the 53 pupils not receiving 1986-87 
CLEAR dropped 5.9, the 5 pupils served gained 8.8 points). The average NCE 
gain from pretest to districtwide test was 7.5 (6.2 for pupils not receiving 
1986-87 CLEAR, 21.0 for the 5 pupils served). Of the 58 first-graders in the 
subsampie, 43.12 (25) maintained or exceeded their posttest NCE level on the 
districtwide test. 

On the districtwide test 44.8% (25) of the 58 pupils in the subsampie 
scored above the 36th percentile (45.3% [24] of the 53 pupils not receiving 
1986-87 CLEAR, 4U.0% [2] of the 5 served). 
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AVERAGE 
NCE 




10-85 4-86 9-86 



TEST DATE 



Figure 3. A comparison of the average pretest, posttest, 
and sustained effects NCE scores for second- 
graders in the 1985-86 Chapter 1 CLEAR program. 

The average pretest-posttest NCE gain l^c second-graders in the Chapter 1 
CLEAR program was 9.9 points. When the sustained effects testing was done in 
September, 1986 the average NCE score dropped 8.0 points. The average NCE gain 
from pretest to sustained effects test was 1.9 points. Of the 630 CLEAR 
second-graders in the study, 32.4% (204 pupils) maintained or exceeded their 
NCE level on the sustained effects test. 

On the pretest 19.2% (121) of the 630 in the Chapter 1 CLEAR program 
scored above the 36th percentile. On the posttest 46.2% (291) scored above the 
36th percentile and 27.6% (174) scored above on the sustained effects test. 
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AVERAGE 
MCE 



52 •• 
48 •• 
44 •• 
40 ■• 
36 •• 
32 ■■ 
28 •■ 
24 •• 
20 •• 
16 •• 
12 ■ 
8 

4 •■ 
0 •- 



2ND GRADE 1985-86 CLEAR 



^ 




1986-87 CLEAR (n=193) 
•°- NO 86-87 CLEAR (n»385) 
•■" TOTAL (n»578) 



10-85 4-86 9-86 

TESPDATE 



4-87 



■• 


52 


•• 


48 




44 




40 




36 




32 




28 




24 




20 




16 




12 




8 




4 




0 



Figure 3a. A comparison of the avsrage pretest, posttest, 
sustained effects, and districtwide NCE scores 
for second-graders in the 1985-86 Chapter I 
CLEAR program. 

For the subsampie of second-graders in the 1985-86 Chapter 1 CLEAR program 
having a fourth (districtwide) test score, the average pretest-posttest NCE 
gain was 10.7 points. When the districtwide testing was done in April, 1987 
the average NCE score dropped 1.8 points (the 385 pupils not receiving 1986-87 
CLEAR program service dropped 4.8, the 193 pupils served gained 4.1 points). 
The average NCE gain from pretest to districtwide :est was 8.9 (10.2 for pupils 
not receiving 1986-87 CLEAR, 6.3 for those served). Qf the 578 second-graders 
in the subsampie, 47.8% (276) maintained or exceeded their posttest NCE level 
on the districtwide test. 

On the districtwide test 43.1% (249) of the 578 pupils in the subsampie 
scored above the 36th percentile (50.4% [194] of the 385 pupils not receiving 
1986-87 CLEAR orogram service, 28.5% (55 j of the 193 served). 
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10-85 4-86 9-86 



TEST DATE 



Figure 4. A comparison of the average pretest, posttest, 
and sustained effects NCE scores for third-graders 
in the 1985-86 Chapter I CLEAR program. 

The average pretest-posttest NCE gain for third-graders in the Chapter 1 
CLEAR program was 9.7 points. When the sustained effects testing was done in 
September, 1986 the average NCE score dropped 6.6 points. The average NCE gain 
from pretest ro sustained effects test was 3.1 points. Of the 667 CLEAR 
third-graders in the study, 34.5% (230 pupils) maintained or exceeded their NCE 
level on the sustained effects test. 

On the pretest 21. 6i (144) of the 667 in the Chapter I CLEAR program 
scored above the 36th percentile. On the posttest 40.5% (270) scored above the 
3bth percentile and 28.6% (191) scored above on the sustained effects test. 
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AVERAGE 
NCE 



52 T 
48 
44 
40 
36 
32 + 
28 
24 
20 
16 •• 
12 
8 •• 
4 




4.. 
0 +■ 



1986-87 CLEAR {n.346) 
•°- NO 86-87 CLEAR {n.245) 
•■- TOTAL {n.591) 



10-85 4-86 9-86 

TEST DATE 



52 
48 
■• 44 
•■ 40 
■■ 36 
32 
28 
24 
20 
16 
12 
■■ 8 
., 4 

0 



4-87 



Figura 4a, 



for third-graders in the 1985-86 Chapter 1 
CLEAR program. 

For the subsampie of third-graders in the IQflS aa 
having a fourth (districtwidej test score ^hi ^^^'^ ^ ^^^^ P^S"-" 

gain was 9,8 points. When che district^J^e'testlnr"^! Pretest-posttest NCE 
average NCE score was up 0,3 points Vthr ^rs ..^ ^" ^P""' ^987 the 

program service dropped 1 0, the 346 P^^'^^"""^ receiving 1986-87 CLEAR 

average NCE gain from pretest to districtwlH. T ''^ P"^"'^^^- The 

noc receiving 1986-87 CLEAR in n "^^st was 10,1 (10,2 for pupils 

in the subsampie, 41,0% (f63) m;inr^ . " «^ '^'^^ 591 third-graders 

on the districtwide test, "--^"-^^^"e^ or exceeded their post test NCE level 

scorerabt\r^3'6rptclrt\ir ^^^^^ -^--^^ 

1986-67 CLEAR program service!'3:,4r[\'f9] of ^th:S:6%ervedr''^ 
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Figure 5. A comparison of the average pretest, oosttest 
and sustained effects NCE scores for fourth- ' 
graders in the 1985-86 Chapter 1 CLEAR program. 

CLE^'toZl^llsT^^^^^ fourth-graders in the Chapter I 

September. 1986 the average ncE score droDoed'l y'" f'"'^'' '^"'"^ '^'^ 
fro, pretest to sustainfd effects" ast'^^^^^^ The average NCE gain 

fourth-graders in the study 34 62 "^s 0 q points. Of the 688 CLEAR 

NCE level on the sustained effect's test" maintained or exceeded their 

scored^-ah^'ve t^h^e^Sth pifent^^'^ t ill ^" T ^^^^"^ ^ P"^"™ 

3-h percentile and 28^3. (-^^s-cor^eVal^v^r ThU^^^^^^^^ 
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Figure 5a. A comparison of the average pretest, posttest, 
sustained effects, and districtwtde NCE scores 
for fourth-graders in the 1985-86 Chapter 1 
CLEAR program. 

havinf a 'LrTrdittril'^;;'.'"^"'"' '""^ '^'^''^ ^'^^P'^" ' ^^^^ P-^^^^am 
nayxng a fourth (dis'.rictwide) test score, the average pretest-posttest ifE 

gain was 4.5 points. When the districtwide testing „as done in ApriT 1987 t£e 
cllll^" I score dropped 1.9 points (the 279 p'upiis not receiving 986-?/ 

ihe average NCE gain from pretest to districtwide test was 2.7 (3 1 for n.-mil 
not receiving 1986-87 CLEAR, 2.3 for those served). Of the 642 fourth-g'ade s 

On the districtwide test 31.22 (200) of the 642 pupils in the suK.amDle 
mr.Sl'cLZ"''' percentile (43.0% [120] of the 279%upi;s no^ UI" ng 

1986-87 CLEAR program service, 22.0% [80] of the 363 served). 



21 

ERjC EVALSRVCS/P530/SUSRPT01 



16 



AVERAGE 



52 
48 
44 
40 
36 
32 
28 
24 
20 
16 
12 
8 
4 
0 



5TH GRADE CLEAR 



i 



i 



1985-86 CLEAR (n=:597) 





r 52 




■ 48 




• 44 




■ 40 




■ 36 




■ 32 




■ 28 




■ 24 




' 20 




• 16 




■ 12 




■ 8 




■ 4 

• 0 



10*85 4*86 9-86 

TEST DATE 



Figure 6» A comparison of the average pretest, posttest, 
and sustained effects NCE scores for fifth-graders 
in the 1985-86 Chapter 1 CLEAR program. 

The average pretest-posttest NCE gain for fifth-graders in the Chapter I 
CLEAR program was 3.0 points. When the sustained effects testing was done in 
September, 1986 the average NCE score dropped 4.5 points. The average NCE gain 
from pretest to sustained effects test was -1.5 points. Of the 597 CLEAR 
tifth-graders in the study, 35. 8X (214 pupils) maintained or exceeded their NCE 
level on the sustained effects test. 

On the pretest 21. 4Z U28) of the 597 in the Chapter 1 CLEAR program 
scored above the 36th percentile. On the posttest 21.1% (126) scored above the 
36th percentile and 16.6/i (99) scored above on the sustained effects test. 
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Figure 6a. A comparison of the average pretest, posttest, 
sustained effects, and dlstrlctwlde NCE scores 
for fifth-graders In the 1985-86 Chapter 1 
CLEAR program. 

For the subsampie of fifth-graders In the 1985-86 Chapter 1 CLEAR program 
having a fourth (dlstrlctwlde) test score, the average pretest-posttest NCE 
gain was 3.0 points. When the dlstrlctwlde testing was done in April, 1987 the 
average NCE score dropped 1.2 points (the 241 pupils not receiving 1986-87 
CLEAR program service dropped 2.4, the 293 pupils served dropped 0.1 points). 
The average NCE gain from pretest to dlstrlctwlde test was 1.9 (1.7 for pupils 
not receiving 1986-87 CLEAR, 2.0 for those served). Of the 534 fifth-graders 
in the subsampie, 41.6% (222) maintained or exceeded their posttest NCE level 
on the dlstrlctwlde test. 

On the dlstrlctwlde test 26.4% (141) of the 534 pupils in the subsampie 
scored abo-e the 36th percentile (.39.4% [95J of the 241 pupils not receiving 
1986-a7 CLEAR program service, 15. 7X [46] of the 293 served). 
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Figure 7, A comparison of the average pretesc, posttest, 
and sustained effects NCE scores for sixth-graders 
in the 1985-86 Chapter I CLEAR program. 

average pretest-posttest NCE gain for sixth-graders in the Chapter 1 
CLEAR program was 4.4 points. When the sustained effects testing was done in 
September, 1986 the average NCE score dropped l.O points. The average NCE gain 
from pretest to sustained effects test was 3.4 points. Of the 460 CLEAR 
sixth-graders in the study, 49.8X (229 pupils) maintained or exceeded their NCE 
level on the sustained effects test. 

On the pretest 14.8Z (68) of the 460 in the Chapter 1 CLEAR program scored 
above the 36th percentile. On the posttest 20.4% (94) scored above the 36th 
percentile and 23. OX (106) scored above on the sustained effects test. 



ERIC 

EVALSRVCS/P530/SUSRPT01 



24 



19 



6TH GRADE 1985-86 CLEAR 



AVERAGE 
NCE 



52 
48 
44 
40 
36 •• 
32 •• 
28 •■ 
24 •■ 
20 •■ 
16 ■■ 
12 ■• 
8 •• 
4 <• 
0 




1986-87 CLEAR (na77) 
•°- NO 86-87 CLEAR (na337) 
•■■ TOTAL (n-414) 



■ 1 > 

10-85 ^.-86 9-86 

TEST DATE 



4-87 





52 




48 




44 




40 




36 




32 




28 




24 




20 




16 




12 




8 




4 




0 



ERIC 



Figure 7a. A comparison of the average pretest, posttest, 
sustained effects, and districtwide NCE scores 
for sixth-graders in the 1985-86 Chapter 1 
CLEAR program. 

For the subsample of sixth-graders in the 1985-86 Chapter 1 CLEAR program 
having a fourth (districtwide) test score, the average pretest-posttest NCE 
gain was 4.5 points. When the districtwide testing was done in April, 1987 the 
average NCE score rose an additional 5.3 points (the 337 pupils not receiving 
1986-87 CLEAR program service gained 4.9, the 77 pupils served gained 7.2 
points;. The average NCE gain from pretest to districtwide test was 9.8 (10.3 
for pupils not receiving 1986-87 CLEAR, 7.5 for chose served). Of the 414 
sixth-graders in the subsample, 71.0% (294) maintained or exceeded their 
posttest NCE level on the districtwide test. 

On the districtwide test 45.9% (190) of the 414 pupils in the subsample 
scored above the 36th percentile (49.0% [165] of the 337 pupils not receiving 
1986-87 CLEAR program service, 32.5% [25] of the 77 served). 
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Figure 8. A comparison of the average pretest, posttest, and 
sustained effects NCE scores for seventh-graders 
in the 1985-86 Chapter I CLEAR program. 

The average pretest-posttest NCE gain for seventh-graders in the Chapter 1 
CLEAR program was 6.4 points. When the sustained effects testing was done in 
September, 1986 the average NCE score dropped 7.6 points. The average NCE gain 
from pretest to sustained effects test was -1.2 points. of the 148 CLEAR 
seventh-graders in the study, 25.0% (37 pupils) maintained or exceeded their 
NCE level on the sustained effects test. 

On the pretest 25.7% (38) of the 148 in the Chapter I CLEAR program scored 
above the 36th percentile. On the posttest 45.9% (68) scored above the 36th 
percentile and 25.0% (37) scored above on the sustained effects test. 
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Figure 8a. A comparison ot the average pretest, posttest, 
sustained effects, and districtwide NCE scores 
for seventh-graders in the 1985-86 Chapter 1 
CLEAR program. 

For the subsample of seventh-graders in the 1985-86 Chapter 1 CLEAR 
program having a fourth (districtwide) test score, the average pretest-posttest 
NCE gain was 6.4 points. When the districtwide testing was done in April, 1987 
the average NCE score dropped 7.3 points (the 124 pupils not receiving 1986-87 
CLEAR program service dropped 7.6, the 6 pupils served dropped 1.7 points). 
The average NCE gain from )retest to districtwide test was -0.9 (-0.8 for 
pupils not receiving 1986-87 CLEAR, -3.8 for those served). Of the 130 
seventh-graders in the subsample, 23.1% (30) maintained or exceeded their 
posttesc NCE level on the districtwide test. 

On the districtwide test 23.8% (31) of the 130 pupils in the subsample 
scored above the 36th percentile (25.0% [31] of the i.a4^pupils not receiving 
1986-87 CLEAR program service, 0.0% (OJ of the 6 served). 
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Figure 9. A comparison of the average pretest, posttest, 
and sustained effects NCE scores for ail pupils 
in the 1985-86 Chapter 1 CLEAR and Reading 
Recovery programs. 

The average pretest-posttest NCE gain for all pupils in the Chapter 1 
CLEAR program was 6.5 points. When the sustained effects testing was done in 
September, 1986 the average NCE score dropped 5.3 points. The average NCE gain 
from pretest to sustained effects test was 1.2 points. of the 3338 CLEAR 
pupils in the study, 36. 2Z (1134 pupils) maintained or exceeded their NCE level 
on the sustained effects test. 

On the pretest 23.0% (767) of the 3338 in the Chapter 1 CLEAR program 
scored above the 36th percentile. On the posttest 36. OZ (1203) scored above 
the 36th percentile and 25,5% (852) scored above on the sustained effects test. 
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Figure 9a. A comparison of the average pretest, posttest, 
sustained effects, and districtwide NCE scores 
for pupils in the 1985-86 Chapter 1 CLEAR and 
Reading Recovery programs. 

For the subsample of pupils in the 1985-86 Chapter 1 CLEAR program having 
a fourth (districtwide) test score, the average pretest-posttest NCE gain was 
6.7 points. When the districtwide testing was done in April, 1987 the average 
NCE score dropped 0.6 points (the 1684 pupils not receiving 1986-87 CLEAR 
program service dropped 2.1, the 1305 pupils served gained 1.3 points). The 
average NCE gain from pretest to districtwide test was 6.1 (6.8 for pupils not 
receiving 1986-87 CLEAR, 5.1 for those served). Of the 2989 pupils in the 
subsample, 47. 9X (1432) main .ained or exceeded their posttest NCE level on the 
districtwide test. 

On the districtwide test 37.9:? '1134) of the 2989 pupils in the subsample 
scored above the 36th percentile (4/. 4% 1799] of the 1684 pupils not receiving 
1986-87 CLEAR program service, 25.7% [335 J of the 1305 served). 
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Table 3 

NCE Means and Standard Deviations 
in Reading for Pupils in Each of 
Three Testing Periods 



1985-86 Grade/ N Pretest (10-85) 
Treatment Avg, (SD) 



1 


CLEAR 


55 


44.2 


(13.9) 




Recovery 


93 


31.1 


(15.7) 


2 


CLEAR 


630 


33.2 


(10.2) 


3 


CLEAR 


667 


31.7 


(13.8) 


4 


CLEAR 


688 


36.1 


(14.1) 


5 


CLEAR 


597 


33.7 


(10.5) 


6 


CLEAR 


460 


32.0 


(11.9) 


7 


CLEAR 


148 


35.6 


(10.4) 



Sustained Effects 
Posttest (4-86) Test (9/86) 



Avg. 


(SD) 


Avg. 


(SD) 


46.1 
41.1 


(12.3) 
(14.6) 


42.0 
36.3 


(13.3) 
(11.9) 


43.1 


(18.4) 


35.1 


(14.9) 


41.3 


(11.4) 


34.7 


(14.3) 


40.8 


(13.1) 


36.1 


(12.1) 


36.7 


(9.4) 


32.2 


(13.0) 


36.4 


(10.7) 


35.4 


(11.0) 


42.0 


(11.6) 


34.4 


(12.3) 



Total 3338 33.6 (12.5) 40.2 (13.3) 34.9 (13.3) 
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Table 3a 

NCE Meaas and Standard Deviations 
in Reading for Pupils in Each of 
Four Testing Periods 



1985-86 Grade/ 
Treatment 



N 



Pretest 
(10-85) 
Avg> (SD) 



Posttest 
(4-86) 
Avg> (SD) 



Sustained 
Effects 
Test (9-86) 
Avg> (SD) 



1986-87 
CLEAR 
Service 



Districtwide Test (4-87 ) 



N Avge (SD) 



No 86-87 
CLEAR 
Service 



N Avg> (SD) 



Total 



1 


CLEAR 
Recovery 


42 
58 


43.8 
31.0 


(14.7) 
(17.7) 


45.5 
43.2 


(12.2) 
(13.8) 


41.5 
37.5 


(13.2) 
(13.0) 


22 
5 


37.6 
36.2 


(19.8) 
(11.5) 


20 
53 


49.3 (15.3) 
38.8 (21.6) 


42 
58 


43.1 (18.5) 
38.5 (20.9) 


2 


CLEAR 


578 


33.2 


(10.3) 


44.0 


(18.3) 


35.6 


(14.5) 


193 


38.7 


(11.3) 


385 


43.9 (11.5) 


578 


42.1 (11.7) 


3 


CLEAR 


591 


32.2 


(13.7) 


42.0 


(10.9) 


35.6 


(14.2) 


346 


39.2 


(13.9) 


245 


46.6 (14.2) 


591 


42.3 (14.5) 


4 


CLEAR 


642 


36.4 


(14.1) 


41.0 


(13.0) 


36.3 


(12.2) 


363 


36.6 


(9.6) 


279 


42.4 (10.4) 


642 


39.1 (10.4) 


5 


CLEAR 


534 


33.9 


(10.5) 


36.9 


(9.5) 


32.6 


(12.7) 


293 


32.8 


(9.8) 


241 


39.4 (11.3) 


534 


35.8 (11.0) 


6 


CLEAR 


414 


32.0 


(11.9) 


36.5 


(10.5) 


35.7 


(11.1) 


77 


39.0 


(9.2) 


337 


42.5 (11.8) 


414 


41.9 (11.4) 


7 


CLEAR 


130 


35.5 


(10.7) 


41.9 


(11.2) 


34.3 


(12.3) 


6 


33.5 


(7.3) 


124 


34.6 (13.0) 


130 


34.6 (12. P.) 



Total 



2989 



33.9 (12.5) 40.6 (13.2) 35.3 (13.1) 1305 36.9 (11.6) 1684 42.3 (12.7) 2989 40.0 (12.6) 
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Table 4 



Percent of Pupils Who Scored 
Above the 36th Percentile in 
Reading for Each of Three 
Testing Periods 



1985-86 Grade/ 
Treatment 


N 


Pretest 
(10-85) 


Posttest 
(4-86) 


Sustained Effects 
Test (9-86) 


1 CLEAR 


55 


52.7% 


58.2% 


49.1% 


Recovery 


93 


24,7% 


50.5% 


24.7% 


2 CLEAR 


630 


19.2% 


46.2% 


27.6% 


3 CLEAR 


667 


21.6% 


40.5% 


28.6% 


4 CLEAR 


688 


31.4% 


40.0% 


28.3% 


5 CLEAR 


597 


21.4% 


21.1% 


16.6% 


6 CLEAR 


460 


14.8% 


20.4% 


23.0% 


7 CLEAR 


148 


25.7% 


45.9% 


25.0% 


Total 


3338 


23.0% 


36.0% 


25.5% 
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Table 4a 



Percent of Pupils Who Scored 
Above the 36th Percentile in 
Reading for Each of Four 
Testing Periods 



Districtwide Test (4-87) 



1985-86 Grade/ 
Treatment 


N 


Pretest 
(10-85) 


Posttest 
(4-86) 


Sustained 
Effects 
Test (9-86) 


1986-87 
CLEAR 
(N) % 


No 86-87 
CLEAR 
(N) % 


Total 
(N) % 


1 


CLEAR 


42 


54.8% 


54.8% 


47.6% 


22 


36.4% 


20 


70.0% 


42 


52.4% 




Recovery 


58 


29.3% 


51.7% 


27.6% 


5 


40.0% 


53 


45.3% 


58 


44.8% 


2 


CLEAR 


578 


19.2% 


48.3% 


28.5% 


193 


28.5% 


385 


50.4% 


578 


43.1% 


3 


CLEAR 


591 


22.2% 


42.1% 


30.8% 


346 


34.4% 


245 


63.7% 


591 


46,5% 


4 


CLEAR 


642 


31.8% 


40.0% 


28.7% 


363 


22.0% 


279 


43.0% 


642 


31.2% 


5 


CLEAR 


534 


21.9% 


21.7% 


17.0% 


293 


15.7% 


241 


39.4% 


534 


26.4% 


6 


CLEAR 


414 


15.5% 


21.0% 


?4.4% 


77 


32.5% 


337 


49.0% 


414 


45.9% 


7 


CLEAR 


130 


26.2% 


44.6% 


25.4% 


6 


0.0% 


124 


25.0% 


130 


23.8% 


Total 




2989 


23.5% 


36.8% 


26.5% 


1305 


25.7% 


1684 


47.4% 


2989 


37.9% 
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Table 5 



Summary of NCE Average Changes 
and The Percent of Pupils Who 
Maintained Their Achievement Level 
(Three Testing Periods) 



Maintained 



1985-86 
Grade/Treatment 


N 


Pretest (10-85) to 
Posttest (4-86) 


Posttest (4-86) to 
Sustained Effects (9-86) 


Pretest (10-85) to 
Sustained Effects (9-86) 


Posttest (4-86) 
NCE Level 
by (9-86) 


1 


CLEAR 
Recovery 


55 

93 


1.9 

10.0 


-4.1 
-4.7 


-2.2 
5.3 


40.0% 
38.7% 


2 


CLEAR 


630 


9.9 


-8.0 


1.9 


32.4% 


3 


CLEAR 


667 


9.7 


-6.6 


3.1 


34.5% 


4 


CLEAR 


688 


4.7 


-4.7 


0.0 


34.6% 


5 


CLEAR 


597 


3.0 


-4.5 


-1.5 


35.8% 


b 


CLEAR 


460 


4.4 


-1.0 


3.4 


49.8% 


7 


CLEAR 


148 


6A 


^7^6 







Total 


3338 


6.5 


-5.3 


1.2 


25.0% 

36.2% 
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Table 5a 

Suiroary of wQs Average Changes 
and Percent of Pupils Who 
Maintained Their Achievement Level 
(Four Testing Periods) 



NCE Change 



Posttest(4-86) to Dlstrictwide Test(4-^7) 
Pretest( 10-65) to 1986-87 No 1986-67 
1985-86 Grade/ Pcsutest(4"86) CLEAR CLEAR Total 

'^^''^^^a^^^ N Avg> N Avg, N Avg, N Avg, 



Pretest( 10-85) to Districtwide Test(4-87) 

1986-87 No86=87 
CLEAR CLEAR Total 

N Avg, N Awg. N Avg, 



Maintained Posttest (4-86) 

NCE Level by (4-67) 
1986-87 No 86-87 
CLEAR CLEAR Total 



1 


CLEAR 


42 


1.7 


22 


-1.8 


20 


-3.1 


42 


-2.4 




Recovery 


58 


12.2 


5 


8.8 


53 


-5.9 


58 


^.6 


2 


CLEAR 


578 


10.7 


193 


4.1 


385 


-^.8 


578 


-1.8 


3 


CUEAR 


591 


9.8 


346 


1.3 


245 


-1.0 


591 


0.3 


4 


CUEAR 


642 


4.5 


363 


-0.2 


279 


-4.1 


642 


-1.9 


5 


CLEAR 


534 


3.0 


293 


-0.1 


241 


-2.4 


534 


-1.2 


6 


CLEAR 


414 


4.5 


77 


7.2 


337 


4.9 


414 


5.3 


7 


CLEAR 


130 


6.4 


6 


-1.7 


124 


-7.6 


130 


-7.3 



22 


-3.1 


20 


2.0 


42 


-0.7 


40.9% 


40.0% 


40.S^ 


5 


21.0 


53 


6.2 


58 


7.5 


80.0% 


39.6% 


43.1% 


193 


6.3 


385 


10.2 


578 


8.9 


67.9% 


37.7% 


47.8% 


346 


10.0 


245 


10.2 


591 


10.1 


57.2% 


43.7% 


51.6% 


363 


2.3 


279 


3.1 


642 


2.7 


47.7% 


32.3% 


41.0% 


293 


2.0 


241 


1.7 


534 


1.9 


44.7% 


37.8% 


41.6% 


77 


7.5 


337 


10.3 


414 


9.8 


75.3% 


70.0% 


71.0% 


6 


-3.8 


24 


-0.8 


130 


-0.9 


66.7% 


21.0% 


23.1% 



Total 



2989 6.7 



1305 1.3 1684 -2.1 2989 -0.6 



1305 5.1 



1684 6.8 2989 6.1 



54.3% 43.0% 47.9% 
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Discussion 



The first grade Reading Recovery, second grade CLEAR, and third grade 
CLEAR pupils demonstrated the lar^Tst pretest to posttest gains, an average 
treatment effect of approximately 10 points • By September, 1986, the time of 
sustained effects testing, first grade Reading Recovery pupils showed, on 
average, the largest sustained gain for all grade levels, a gain of 5.3 NCE 
points. Sixth-, third-, and second-grades had respectively 3»4, 3»1, and U9 
average NCE gains for October, 1985 to September, 1986# Nearly half (49#8%) of 
the sixth grade pupils maintained or exceeded their posttest level on the 
sustained effects test, whereas 38»7% of first grade Recovery, 34 •5% of third 
grade, and 32*4% of second grade pupils did so (see Table 5). 

Looking further, to April, 1987 , sizable average NCE gains from the 
October, 1985 pretest to the districtwide test are observed for four grade 
levels: 10»1 for third grade, 9.8 for sixth, 8.9 for second, and 7.5 for first 
grade Reading Recovery. Seventy-one percent (71 #0%) of the sixth grade 
students maintained or exceeded their posttest level on the districtwide test* 
At third grade 51»6%, second grade 47 #8%, and first grade Recovery 43*1% did so 
(see Table 5a) • 

Some differential gain is noted for the students who were served by CLEAR 
the following 1986-87 school year and those who were not. This is expected in 
that Chapter 1 CLEAR services are targeted to pupils who are not achieving well 
in reading (see Amorose et»al [1986] for a detailed discussion of multiple year 
Chapter 1 CLEAR service patterns and reading achievement over a five-year 
period) • 

Comparing the overall NCE data with the 1983-84 study, a much larger 
decline from spring to fall was noted in the 1985-86 data* Specifically the 
1982-83 study indicated a greater "pre-post" gain and then a smaller 
"post-sustained" decline than Figure 9 of the current study* An inspection of 
the 1985-86 data reveals that first grade and fifth grade CLEAR pupils scored 
higher on the pretest and did not produce the gains observed in 1982-83* 
Further inspection reveals that grades 6, 3, and 2 produced sustained gains 
higher than the other grades' gains - similar to the 1982-83 performance* 
However, grades 2, 3, and 7 showed greater decx^ne from posttest to sustained 
effects test* 

Summary 

In accordance with the Chapter 1 Federal guidelines the Columbus Public 
Schools completed the sustained effects study* The results of this study will 
be used in planning Chapter 1 programs* 

The sustained effects design included a fall-spring-fall testing schedule 
with an additional spring test for a subsample of pupils* In the 1985-86 
Chapter 1 CLEAR and Reading Recovery programs 3338 pupils were in the 
evaluation sustained effects group, 2989 in the subsample* Those pupils took 
each of the 3 or 4 scheduled tests, met the 80 percent attendance criterion of 
instructional days, and were promoted to the next grade level* 



EVALSRVCS/P530/SUSRPT01 4 0 



31 



The results of the sustained effects study indicate some decline in all 
grade levels in reading during the summer months, with sixth grade having the 
smallest decline. The greatest net gains from Fall 1985 to Fall 1986 were 
noted for first grade Reading Recovery, and 2nd, 3rd, and 6th grade CLEAR* 
Pupils from these grade le^^els then substantially improved their Fall 1986 
achievement levels by Spring 1987* 
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